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Belief shouldn’t be On / Off 


Degrees of belief 
Confidence intervals 
Credence levels 

What's the likelihood of smarter than 
HLI by 2050? Or 2075? Or 2100? 










Intelligence is powerful 




Intelligence for the most part got our ancestors from mid-level 
scavengers to apex predators, from hand axes —> the international 
L115A3 sniper rifle —► fully automated drones, from barefoot 
meandering —> heavier than air flight —> escape velocity —> landing 
robots on mars and beyond.. 


Without giving precise numbers, I assume that if you substitute our 
human minds with better-designed minds, running at 1 million times 
human clock speed, we would see a much faster rate of progress in 
technoscience. 



The space of possible minds/intelligences 


Human intelligence is 
a small fraction in the 
much larger mind 
design space 
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The space of possible minds 



Bipping Als 


Minds-in-general 


Posthuman mindspace 


Freepy Als 


Gloopy Als 


Transhuman mindspace 


Human minds 


Humans, not being the only possible 
mind, are similar to each other 
because of the nature of sexual 
reproduction 

Transhuman minds, while still being 
vestigially human, could be smarter or 
more conscious than unaugmented 
humans 

Posthuman minds may exist all over 
the spectrum - occupy a wider 
possibility space in the spectrum 
It may be very hard to guess what an 
Al superintelligence would be like, 
since it could end up anywhere in this 
spectrum of possible minds 








Full understanding of human intelligence not required for Al 


• But it would certainly help 

• For most of human history, we 
didn’t really understand fire - 
but made good use of it 

• Today we don’t fully 
understand biology, though we 
are able to develop effective 
medicine 

• We don’t fully understand the 
brain - have developed 
intelligent algorithms - some 
neuromorphic (brain inspired) 
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Evolution vs Intelligent Design of Intelligent Processes 


Human intelligence, the result of "blind" evolutionary 
processes, has only recently developed the ability for 
planning and forward thinking - deliberation. On the other 
hand, the rest and almost all our cognitive tools were the 
result of ancestral selection pressures, forming the roots of 
almost all our behavior. As such, when considering the design 
of complex systems where the intelligent designer - us - 
collaborates with the system being constructed, we are 
faced with a new signature and a different way to achieve 
AGI that's completely different than the process that gave birth 
to our brains. 



AGI vs Narrow Al 

Strong Al or "Artificial General 
Intelligence" (AGI) is Al that has the 
ability to perform "general intelligent action". 



Distinction: Narrow Al (or Weak Al/Applied Al) used to accomplish specific problem 
solving or reasoning tasks that do not encompass (or in some cases are completely 
outside of) the full range of human cognitive abilities. 


Gradient between Narrow & General - what lies in between? 


Narrow 

Al 
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• Narrow: Deep Blue (Chess), Watson (Jeopardy) 

• In between? AlphaGo? AlphaStar? Us? Dolphins, Bonobos? 

• Optimally General: Computable AIXI? 






Peaks & Valleys in Cognitive capability 

Peaks of Al cognitive 
capabilities have already 
crossed over valleys in 
human cognitive capabilities. 

Will we see a rising tide in 
general Al cognitive 
capability? 

Will the valleys in Al cognitive 
ability rise above the peaks in 
human cognitive capability? 





Al Capability: Optimal 

• Tic-Tac-Toe 

• Connect Four : 1988 

• (aka 8x8 draughts): Weakly solved (2007) 

• Rubik's Cube : Mostly solved (2010) 

• : Statistically optimal in the sense that "a human lifetime of play is not sufficient to establish with 

statistical significance that the strategy is not an exact solution" (2015) - see vvv.deeostack.ai/ 



Search spaces very small - though Heads-up limit hold’em poker is interesting - like most other NN 
style Al - the majority of the behaviour was learned via self play in a reinforcement learning 
environment 











State of play in Al Progress - Some areas already Super-human 


Al Capability: Super-human 

• Othellc (aka reversi): c. 1997 

• Scrabble : 2006 

• Backgammon : c. 1995-2002 

• : Supercomputer (c. 1997); Personal computer (c. 2006); Mobile 
phone (c. 2009); Computer defeats human + computer (c. 2017) 

• Jeopardy! : Question answering , although the machine did not use 

speech recognition (2011) 

• Shogi : c. 2017 

• Arimaa : 2015 

• 3o: 2017 <- AlphaGo - learned how to play from scratch 

• Heads-up no-limit hold'em poker: 2017 


Board games and some computer games are well defined 
problems and easier to measure competence against. 

























Al Capability: High-human 

• Crosswords : c. 2012 

• Dota 2 : 2018 

• card-playing: According to a 2009 review, "the best programs are 
attaining expert status as (bridge) card players", excluding bidding. 

• jtarCraft :2019 <- AlphaStar 

AiphaGo vs AiphaStar - Unsupervised Learning 

Wider search spaces - Dota 2 and Starcraft - incomplete 
information, long term planning. AlphaStar makes use of 
‘autoregressive policies’.. 



A graphical representation of AlphaStar s processing The system sees whole map from the top down and predicts what 
behavior will lead to victory | Image DoepMmd 


Al Capability: Par-human 

• Optical character recognition for ISO 1073-1:1976 and similar special characters. 

• Classification of images 

• 
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AlphaZero / AlphaStar 

If you were a blank slate, and were given a GO board, and were told you had 24 hours to 
teach yourself to become the best Go player on the planet, and you did exactly that, and 
then you did the same thing with Chess and Shogi (both more complex than Chess), and 
later Starcraft II, and then Protein Folding - wouldn't you say that proved you had the ability 
to adapt to novel situations? Well AlphaZero did all that and it didn't use brute force to do it 
either. 

Stockfish is another Chess program but unlike AlphaZero it didn't teach itself humans did. 
Stockfish could easily beat any human player but it couldn't beat AlphaZero despite the fact 
that Stockfish was running on a faster computer and could evaluate 70,000,000 positions a 
second while AlphaZero's much smaller computer could only do 80,000. 

And Al is becoming less narrow and brittle every day. GO is played on a 19 by 19 grid, but if 
you changed it to 20 by 20 or 18 by 18 and gave it another 24 hours to teach itself AlphaZero 
would be the best player in the world at that new game too - without any human help. 
AlphaZero is not infinitely adaptable, but then humans aren't either. 



Al Capability: Sub-human 

• Optical character recognition for printed text (nearing par-human for Latin-script typewritten text) 

• Object recognition 

• Facial recognition : Low to mid human accuracy (as of 2014) 

• Visual question answering, such as the VQA 1.0 

• Various robotics tasks that may require advances in robot hardware as well as Al, including: 

o Stable bipedal locomotion: Bipedal robots can walk, but are less stable than human walkers 
(as of 2017) 

o Humanoid soccer 

• Speech recognition : "nearly equal to human performance" (2017) 

• Explainabii . Current medical systems can diagnose certain medical conditions well, but cannot 
explain to users why they made the diagnosis. 

• Stock market prediction : Financial data collection and processing using Machine Learning 
algorithms 

• Various tasks that are difficult to solve without contextual knowledge, including: 

o Translation 

o Word-sense disambiguation 
o Natural language processing 


Though Al is classed as par or sub-human, for most of these tasks Al can 
achieve wider capabilities far faster than humans can. 













Below is an example of a cut down version of GPT-2 


Prompt: 

Generated 

completion: 


When will Al surpass human level intelligence? 

A couple of years from now, probably not, maybe as early as 2020. 

Is it possible to control Al with Al? 

Absolutely, it's not hard, and it's not that hard. It's not like controlling 
an oil refinery or something. Al could be programmed to do anything 
you can program a computer to do. It's just that there wouldn't be any 
need for a human in the room, right? For example, if I told Al "You can 
do anything you want, just don't be stupid and you won't be a 
problem," then that could theoretically result in a very complex 
algorithm that I could write in a language that I can't read. And so I'd 
have a very complicated, expensive algorithm that could make me 
millions of dollars. So I would need a human right there at the table 
who wouldn’t try to stop it and I would actually have a very simple, 
very elegant algorithm. That's exactly how it is. 

Is a self-aware Al possible? 

Yes. Self-aware Al is 


https://talktotransformer.com 


TalkToTransformer.com 


QpenAI’s GPT-2 

Better Language Models 
and Their Implications 

“We’ve trained a large-scale unsupervised language 
model which generates coherent paragraphs of text, 
achieves state-of-the-art performance on many 
language modeling benchmarks, and performs 
rudimentary reading comprehension, machine 
translation, question answering, and 
summarization—all without task-specific training.” 

Models can be used to generate data by successively 
guessing what will come next, feeding in a guess as 
input and guessing again. Language models, where 
each word is predicted from the words before it, are 
perhaps the best known example: these models power 
the text predictions that pop up on some email and 
messaging apps. Recent advances in language 
modelling have enabled the generation of strikingly 
plausible passages, such as QpenAI’s GPT-2 . 






Transfer Learning 

Knowledge in one domain is applied to another 

1. I MPALA simultaneously performs multiple tasks—in this case, playing 57 
Atari games—and attempts to share learning between them. It showed signs of 
transferring what was learned from one game to another. 

https://www.technoloavreview.eom/f/610244/deeDminds-latest-ai-transfers-its-learnina-to-new-tasks/ 

2. EGAL - key idea is to combine a neural network policy with a genetic 
algorithm, the Biased Random-Key Genetic Algorithm (BRKGA)..REGAL 
achieves on average 3.56% lower peak memory than BRKGA on previously 
unseen graphs, outperforming all the algorithms we compare to, and giving 
4.4x bigger improvement than the next best algorithm. 

https://deepmind.com/research/publications/REGAL-Transfer-Learnina-For-Fast-Optimization-of-Computation-Graphs 






Getting closer to solving key building 
blocks of Al 



Unsupervised Learning (without 
human direction on unlabelled data) - 
DeepMind’s AlphaGo, AlphaStar, now 
in many other systems 
Transfer Learning (knowledge in one 
domain is applied to another) - 


Recap.. 

1. Intelligence is powerful 

2. Human Intelligence is a small fraction in the space of 
possible mind design 

3. We already have Al smarter than HLI (too many 
examples to list) 

a. Al has surged past HLI in specific domains 

4. Some Al is becoming more general (AGI) 

a. Unsupervised learning 

b. Transfer learning 



The Technological Singularity 

Vernor Vinge coined the term 

"Technolgical Singularity 
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Vernor postulated that 
"Within thirty years, we will have the technological 
means to create superhuman intelligence. Shortly 
after, the human era will be ended." - vernor vinge 
1994 





Intelligence Explosion 

"Let an ultraintelligent machine be defined as a machine 
that can far surpass all the intellectual activities of any man 
however clever. Since the design of machines is one of 
these intellectual activities, an ultraintelligent machine 
could design even better machines; there would then 
unquestionably be an ‘intelligence explosion,’ and the 
intelligence of man would be left far behind. Thus the first 
ultraintelligent machine is the last invention that man need 
ever make." -1. J. Good (1960s) 



Intelligence Explosion 

The purest case - Al rewriting its own source code. 

The key idea : 

• Improving intelligence accelerates 

• Tipping point. Like trying to balance a pen on one end - 
as soon as it tilts even a little, it quickly falls the rest of 
the way. 




So., why be concerned? 


• Such force multipliers (i.e. discoveries in multiple 
areas of science and technology) are increasing the 
likelihood of the sorts of conditions for an intelligence 
explosion to occur 

o An intelligence explosion would have profound 
effects on all us 



Closing the Loop - Positive Feedback 


Positive Feedback loop is 
completed, through an agents 
ability to intelligently design and 
recreate its ability to intelligently 
design and recreate itself! 

Bottle-neck? Other entities? Substrate 
expandable (unlike brain), grey box or 
ideally white box source code which can 
be engineered and manipulated (brain, 
black box code with no manual... 
possibly can be improved through 
biotech...not trivial) 



Al optimizes / Better source 

improves its code leads to 

source code better Al* 



^Smarter Al will be even better at 
improving its source code 




Al: Why should we be concerned? 

• Value Alignment 

“The concern is not that [AGI] would hate or resent us for enslaving it, 
or that suddenly a spark of consciousness would arise and it would 
rebel, but rather that it would be very competently pursuing an 
objective that differs from what we really want.” - Nick Bostrom 

• Race Conditions 

Intelligence is powerful - first to create powerful AGI gets massive 
first mover advantages - where safety measures are discarded in 
favor of speeding up development 



Al: Why should we be concerned? 


The Artilect War 

o Cosmists - those who favor building AGI 
o Terrans - those who don’t 

believe that the ideological disagreements between these two 
groups on this issue will be so strong, that a major "artilect" war, 
killing billions of people, will be almost inevitable before the end of 
the 21st century." - Hugo de Garis 



Moore’s Law: the Colloquial, the Actual, & the Ridiculous 


Though Moore’s law has a specific definition, the term has been used colloquially to indicate a 
general trend of increasing computing power. 


• CPU - transistors per cpu not doubling? 

• GPU - speed increases 

• Economic drivers motivated by general 
advantages to having faster computers. 

• Even if slowdown - small advantages in 
speed can aid competitors in race 
conditions. Though usually surges in 
speedups follow. Not a smooth curve 
upwards. 

• No, speedups don’t have to be increases in 
numbers of transistors in CPU cores. 

Surface-level abstractions used to make specific 
predictions about when the singularity will 
happen 


Moore’s Law - The number of transistors on integrated circuit chips (1971-2018) 

Moore's law describes the empirical regularity that the number of transistors on integrated circuits doubles approximately every two years. 
This advancement is important as other aspects of technological progress - such as processing speed or the price of electronic products - are 
linked to Moore's law. 


Our World 
in Data 


50,000,000,000 


10,000,000,000 

5,000,000,000 


1,000,000,000 
500,000,000 


100 , 000,000 

50,000,000 


10,000,000 

5,000,000 


1 , 000,000 

500,000 


100,000 

50,000 


10,000 

5,000 


72-core Xeon Phi Centriq 2400 i 


. 2400 AGC2 IPU 

SPARC M7<\ '''•32-core AMD Epyc 

IBM z13 Storage Comroller N X\ ^Apple At 2X Bionic 

18-core Xeon Haswell-E5» ~ T " 

Xbox One main SoC^X _ O v 


8-core Xeon 
Six-core Xeoi. 
Dual-core Itanium 2^ 
Pentium D Presler, 

Itanium 2 wit 
9 MB cache 


«%!i* 

| l&s 


Qualcomm Snapdragon 8cx/SCX8180 
'•HiSilicon Kirin 980 * Apple A12 Bionic 
HlSIlicon Kirin 710 


IB cached 
Itanium 2 Madison 6Mt 

Pentium D Smlthfleld-, 
Itanium 2 McKinley^ 


\POWEj^ 

> »*£&**“• 
Pentium 4 Prescott-2Mt ▼x^Core 2 tX 
AMDK8* ♦ Pent(l 

^ ♦ Pentium III Coppermine ^ 

K6-MI 


- « ▼Quad-core* GPU GT2 Core 17 Skylake K 
♦ V - Uuad-core + GPU Core 17 Haswell 

^Apple A7 (dual-core ARM64 •mobile SoC) 

17 (Quad) 

- J —- 2ML3 


Duo Wolfdale 3M 

._ Duo Allendale 

'Pentium 4 Cedar Mill 
Prescott 


Pentium Prq^ 
’entiun^ m 




S^tlO 




Motorola 68020^ 
lntel^0286 


!§$♦ ♦ ARM3 
%Si)S 


TMSJ000 Zllog 280 6805 

RCAJ802 lintel 8085 


; 

n/Soi6 


.0,014^4 


008- ^IntelBOBO®' 


# dP d* .<S ?> # .# .<# .<# .# .# # .<# .# „<# x# ^ .c *- 0 Jp ^ 


f v r *>' ^ V <p~ r fc > ~ 


Data source: Wikipedia (https://en.wikipedia.org/wiki/Transistor_count) 

The data visualization is available at OurWorldinData.org. There you find more visualizations and research on this topic. Ucensed under CC-BY-SA by the author Max Roser. 


A plot (logarithmic scale) of MOS transistor counts for microprocessors against dates of introduction, nearly doubling 
every two years. See https://ourworldindata.org/technoloaical-proaress 







How to evaluate evidence? 


• Trend extrapolation - Black to Dark Box 

o Some views on trend extrapolation favor near term AG I 
o Some views don’t 

o Strange how people carve up what’s useful trend 
extrapolation and what isn’t to support their views 

• Inside view - White to Translucent Box 

o Weak - tenable - loose qualitative views on issues with 
lopsided support 


o Strong - difficult 

• Appeal to authority :D 

Pick your authority! 


“A wise man proportions / 
his belief to the evidence.” ^ 


- David Hume . 



Trend Extrapolation: Accelerating Change 

• Kurzweil's Graphs 
o Not a discrete event 

o Despite being smooth and continuous, explosive once 
curve reaches transformative levels (knee of the curve) 


• Consider the Internet. When the Arpanet went from 10,000 nodes to 20,000 in 
one year, and then to 40,000 and then 80,000, it was of interest only to a few 
thousand scientists. When ten years later it went from 10 million nodes to 20 
million, and then 40 million and 80 million, the appearance of this curve looks 
identical (especially when viewed on a log plot), but the consequences were 
profoundly more transformative. 



Weeeee! 

Enjoy the ride! 

Math and deep insights 
(especially probability) can be 
powerful relative to trend fitting 
and crude analogies. 

Long-term historical trends are 
weekly suggestive of future 
events 

BUT - humans aren’t good at 
math, and aren’t all that 
convinced by it - nice pretty 
graphs or animated gifs seem 
to do a better job. 

Difficulty - convincing humans 
is important - since they will 
drive the trajectory of Al 
development... so convincing 
people of good causes based 
on less-accurate/less-sound 
yet convincing arguments is 
important. 


Exponential Growth of Computing 

Twentieth through twenty first century 
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The IBM Model 350 disk file with a storage space of 5MB from 1956 and a Micro SD Card 


This image is old. 

I saw a 128gb 
SanDisk for $30 
and a $400gb for 
$99 on ebay (circa 
sept 2019) 





SanDisk Ultra 128GB micro SD SDXC Memory Card UHS-I U1 80MB/S Full HD Phone 

Brand new SanDisk 128 GB microSDXC UHS-I 

AU $29.95 ebayplus 

Buy It Now Click & Collect 

+AU $9.99 postage 
^2 Watch 








How Long Until Computers Have the Same Power As the Human Brain? 

Lake Michigan's volume (in fluid ounces) is about the same as our brain's capacity 
(in calculations per second). Computing power doubles every 18 months. At that rate, you see 
very little progress for a long time—and suddenly you're finished. 



The future of exponential technological growth? 




Survey of on timeline of Al 


An expert poll around 2016, conducted by Katja Grace of the Future of Humanity 
Institute and associates, gave median estimates of 3 years for championship Angry 
Birds, 4 years for the World Series of Poker, and 6 years for StarCraft. On more 
subjective tasks, the poll gave 6 years for folding laundry as well as an average 
human worker, 7-10 years for expertly answering 'easily Googleable' questions, 8 
years for average speech transcription, 9 years for average telephone banking, and 
11 years for expert songwriting, but over 30 years for writing a New York Times 
bestseller or winning the Putnam math competition. 


https://aiimpacts.org/miri-ai-predictions-dataset 
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Milestone Best Estimates Without Funding 


Surveys of Al Researchers « 

<D 

1) In 2009,21 Al experts participating o. 
in AGI-09 conference were surveyed . 

Experts believe AGI will occur around 2 

<u 

2050, and plausibly sooner. You can 
see their estimates regarding specific d 
A l achievements: passing the Turing 
test, passing third grade, 
accomplishing Nobel worthy scientific 
breakthroughs and achieving 
superhuman intelligence. 



2010s 2020s 2030s 2040s 2050s 2060s 2070s 2080s 
§ Turing □ Third Grade ■ Nobel 13 Superhuman 


2090s Later 
or 

Never 


2) In 2017 May, 352 Al experts who published at the 2015 NIPS and ICML conferences were surveyed . Based 
on survey results, experts estimate that there’s a 50% chance that AGI will occur before 2060. However, 
there’s significant difference of opinion based on geography: Asian respondents expect AGI in 30 years, 
whereas North Americans expect it in 74 years. 

Some significant job functions that are expected to be automated until 2030 are: Call center reps, truck driving, 
retail sales. 

https://blog.aimultiple.conn/artificial-general-intelligence-singularitv-timing 




















Turing Tests & Winograd Schemas 

• Arguably passed Turing test 

• Winograd Schema success rate of 90% is impressive to people 
in NLP 

• Conversational Al getting better - fast (see earlier example) 
https://talktotransformer.com / OpenAI’s GPT-2 

• Also check out NVIDIAs new sensation :D 

https://www.zdnet.com/article/nvidias-ai-advance-natural-lanauaae-processina- 

aets-faster-and-better-all-the-time 





What won’t Al be able to do within next 50 years? 

Best predictions? 

What Computers Can’t Do - Hubert Dreyfus 
o What Computers Still Can’t Do 

■ Make coffee 

■ Beat humans at physical sport 

■ Write funny jokes? 

■ Write poetry? 

• and novels 

• Formulate creative strategies - the ability to formulate the kind of creative strategies 

that, for instance, define a great lawyer’s ability to form unique arguments or a top CEO to lead his or 
her company in bold new directions. This isn’t just about analyzing data; it’s about venturing into 
unstructured problem-solving tasks, and deciding which pieces of information are relevant and which 
can be safely ignored. 

https://www.diqitaltrends.com/cool-tech/things-machines-computers-still-cant-do-vet/ 



3 Al Boosters 

1. Hardware overhang resulting from 

a. algorithmic performance gains (more efficient algorithms 
require less computing power per unit of value) 

b. or economic gains (M$ donating billion $$ to OpenAI) 

c. Cheaper cloud infrastructure (drops in price/performance of 
already connected cloud) 

2. Self improving systems - feedback loop where gains from 
previous iterations boost performance of subsequent iterations 
of the feedback loop 

3. Economic/Political Will - race to first mover advantage 



Computing Overhang 

• new programming methods use available computing power more efficiently. 

• new / improved algorithms can exploit existing computing power far more efficiently than previous 
suboptimal algorithms... 

o Result: AI/AGI can run on smaller amounts of hardware... This could lead to an intelligence 
explosion , or to a massive increase in the number of AGIs, as they could be easily copied to run 
on countless computers. 

• Examples: 

o In 2010, the President's Council of Advisors on Science and Technology reported on benchmark 
production planning model having become faster by a factor of 43 million between 1988 and 
2003 - where only a factor of roughly 1,000 was due to better hardware, while a factor of 
43,000 came from algorithmic improvements. 

o Sudden advances in capability gains - DeepMind’s AlphaGo, AlphaFold, GPT-2 

• Estimates of time to reach computing power required for whole brain emulation: ~a decade away 

o BUT: very unlikely that human brain algorithms anywhere enar most computationally efficient for 
producing Al. 

o WHY? our brains evolved during a natural selection process and thus weren't deliberately 
created with the goal of being modeled by Al. 


https://wiki.lesswrona.com/wiki/Computing overhang 





Kinetics of an Intelligence Explosion 


Intelligence 
level: Perhaps 
not a continuous 
surge upwards, 
though likely 
would taper off 
way above of 
HLI, and even 
intelligence of 
whole 
civilization. 



Figure 7 Shape of the takeoff. It is important to distinguish between these questions: “Will a 
takeoff occur, and if so, when?" and "If and when a takeoff does occur, how steep will it be?" One 
might hold, for example, that it will be a very long time before a takeoff occurs, but that when it 
does it will proceed very quickly. Another relevant question (not illustrated in this figure) is, "How 
^ ar ge a fraction of the world economy will participate in the takeoff?" These questions are related 
but distinct. 











Bill Gates 


"Reverse Engineering the Brain is Within 
Reach" -“I am in the camp that is concerned 
about super intelligence,” Bill Gates said 
during an “Ask Me Anything" session on 
Reddit . “First, the machines will do a lot of jobs 
for us and not be super intelligent. That should 
be positive if we manage it well. A few 
decades after that, though, the intelligence is 
strong enough to be a concern. I agree with 
Elon Musk and some others on this and don’t 
understand why some people are not 
concerned.” 





‘Expert’ predictions of Never... 


(On Nuclear physics) “The consensus view as expressed by Ernest 
Rutherford on September 11th, 1933, was that it would never be 
possible to extract atomic energy from atoms. So, his prediction was 
‘never,’ but what turned out to be the case was that the next morning 
Leo Szilard read Rutherford’s speech, became annoyed by it, and 
invented a nuclear chain reaction mediated by neutrons! 

Rutherford’s prediction was ‘never’ and the truth was about 16 hours 
later. In a similar way, it feels quite futile for me to make a 
quantitative prediction about when these breakthroughs in AGI will 
arrive.” - Stuart Russell - co-wrote foundational textbook on Al 



Recap.. 

1. Intelligence is powerful 

2. Human Intelligence is a small fraction of a huge space of all possible mind designs 

3. We already have Al smarter than HLI (too many examples to list) - so therefore you can already see Al outperform HLI 

a. Al has surged past HLI in specific domains 

4. Some Al is becoming more general (AGI) 

a. Unsupervised learning 

b. Transfer learning 

5. An Al that could self improve could become very powerful 
very fast 

6. Experts in Al research think that it is likely AGI could be 
developed by mid century 



HLI will rise with Al at the pace of Al? 


• Rapture of the nerds - where HLI will rise with Al by humans 
merging with it - where the true believers will break with human 
history and merge with the machine, while the unaugmented will 
be left in a defiled state... 

o No. 

o Biological bottlenecks - the kernel of human intelligence, 
o Merging with Al enough to keep pace means becoming 
posthuman - defining characteristics of humans will be 
drowned out like a drop of ink in an ocean. 













Navigating the Future 























The future could 
be really ace - we 
have so much to 
look forward to! 













I'LL 


















Extra 

material 










Time to Next Event (years) 


Countdown to Singularity 


io 10 

io 9 

10 8 

10 7 

10 6 

10 5 

10 4 

10 3 

10 2 

10 


Logarithmic Plot 



Life 


10 


10 


Eukaryotic cells, multicellular organisms 

Cambrian Explosion (body plans) 
Reptiles'^— cl Class Mammalia 
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Evolutionary History of Mind 

Daniel Dennet's Tower of Generate and Test 

• "Darwinian creatures", which were simply selected by trial and error on the merits of 
their bodies' ability to survive; its "thought" processes being entirely genetic. 

• "Skinnerian creatures", which were also capable of independent action and therefore 
could enhance their chances of survival by finding the best action (conditioning 
overcame the genetic trial and error of Darwinian creatures); phenotypic plasticity; bodily 
tribunal of evolved, though sometimes outdated wisdom. 

• "Popperian creatures", which can play an action internally in a simulated environment 
before they perform it in the real environment and can therefore reduce the chances of 
negative effects - allows "our Hypothesis to die in our stead" - popper 

• "Gregorian creatures" are tools-enabled, in particular they master the tool of language. - 
use tools (e.g. words) - permits learning from others. The possession of learned 
predispositions to carry out particular actions 


Evolutionary History of Mind 

• To add to Dennet's metaphore: We are building a new 
floor to the tower 

• "Turingian creatures", that use tools-enabled 
tool-enablers, in particular they create autonomous 
agents to create even greater tools (e.g. minds with 
further optimal cognitive abilities) - permits insightful 
engineering of increasingly insightful agents, 
resulting in an Intelligence Explosion. 
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“Those with the privilege 
to know, have a duty to 
act.” - Albert Einstein 












